1. Field of the Invention
This invention relates to a color image processing method and apparatus. This invention particularly relates to a color image processing method and apparatus for a monitor camera, a medical camera, or the like, which is provided with image sensing means, such as charge coupled device (CCD) image sensor.
2. Description of the Prior Art
Color video cameras (hereinbelow referred to simply as the cameras) for recording color images, which cameras are provided with image sensing means, such as CCD image sensors, have heretofore been known. The cameras are currently used widely for broadcasting, for domestic use, as monitor cameras, as medical cameras, and the like.
In cases where a color image is recorded with the camera, if the camera is a single-plate type of camera, in order for color signals (e.g., R, G, and B three primary color signals) to be obtained from a single CCD image sensor, a color filter is superposed upon each of pixels arrayed in a two-dimensional pattern in the color image, and chrominance information is obtained in the form superimposed upon luminance information. A luminance signal and the three primary color signals (or signals in accordance with the three primary color signals) are thus separated from one another. As the color filter, a color filter array, in which three to four kinds of colors are arrayed in a dot-like pattern, is employed.
Each of the three primary color signals having been separated from one another is then subjected to white balance (WB) compensation, gamma (xcex3) compensation for canceling xcex3 characteristics of a cathode ray tube, high-level suppression processing, such as knee processing or white clipping (WC), or the like. The color signals having been obtained from the processing are fed together with the luminance signal into an output circuit for performing matrix processing, encoding processing, or the like. In this manner, the signals are transformed into R, G, and B signals of predetermined levels or video signals conforming to broadcasting standards, such as NTSC standards. As well known, the high-level suppression processing is performed in order to suppress a dynamic range of natural light to a limited signal level range in accordance with the TV signal standards. FIG. 5 is a block diagram showing signal processing in conventional cameras.
With the conventional high-level suppression processing, with respect to each of the three primary color signals, in cases where the signal level of the color signal is higher than a predetermined signal level, the color signal is suppressed. Also, the luminance signal is reproduced in accordance with the three primary color signals, which have been obtained from the suppression processing. In this manner, suppression processing on the luminance signal is performed simultaneously.
As described above, with the conventional high-level suppression processing, with respect to each of the three primary color signals, the suppression processing is performed for each color signal. Therefore, if the signal levels of the color signals are different from one another, it will occur that a certain color signal is suppressed and the other color signals are not suppressed. If a certain color signal is suppressed, the composition ratios of the colors will vary from the original composition ratios. Therefore, in cases where the conventional high-level suppression processing is performed, the problems occur in that the hue changes and the color reproducibility becomes bad (as will be described later with reference to FIGS. 8A, 8B, and 8C). For example, a phenomenon occurs in that, when a photograph of a person is taken, as an aperture is opened, flesh color becomes tinted with yellow. Also, a phenomenon occurs in that, as an aperture is opened, a single-color, high-saturation image becomes tinted with other colors. (Such phenomena will hereinbelow be referred to as the false color phenomenon.) For example, in the cases of endoscope cameras, the region in the body cavity ordinarily has a rugged shape, and it often occurs that a pattern of a dark region and a pattern of a bright region are recorded simultaneously. In the cases of such image sensing conditions, since the dynamic range of the CCD image sensor, or the like, cannot be broadened unlimitedly, it must be selected whether an exposure quantity is to be matched with the dark region and signal components representing the bright region are to be suppressed (e.g., the bright region is rendered white such that its details become imperceptible), or the exposure quantity is to be matched with the bright region and the pattern of the dark region is to be rendered black such that its details become imperceptible. Therefore, the image quality of the pattern of the bright region or the pattern of the dark region becomes bad. Particularly, in cases where the exposure quantity is matched with the dark region and the signal components representing the bright region are suppressed, the phenomenon described below occurs. Specifically, chrominance information of the region in the body cavity is primarily composed of red color information. However, the color filter of the CCD image sensor has the characteristics such that, in such cases, besides the red color signal components, for example, green color signal components are also outputted in no small quantities. Therefore, if the red color signal components are suppressed, the ratio of the green color signal components will become comparatively high. As a result, at a highlight such that it does not become white and its details do not become imperceptible due to the signal suppression, red at a high-saturation area becomes tinted with green.
In order for the false color phenomenon to be prevented, highlight chroma suppression (HLCS) processing may be performed, and the video signals obtained from the processing may be outputted. With the HLCS processing, reference is made to a luminance signal having been separated from detected signals, and processing is performed such that, at a high luminance level such as that at which saturation of the luminance signal is reached, color signals are suppressed so as to eliminate the chrominance information, and only the luminance signal remains.
However, the HLCS processing is the one in which the color signals are suppressed at a highlight area of the luminance signal. Therefore, for example, in cases where the luminance signal is of a low level and only the red color signal is of a high level as in a red high-saturation image, the problems occur in that only the red color signal is suppressed by the high-level suppression processing, and the HLCS processing is not effected. Accordingly, the false color phenomenon occurring due to the suppression of only the red color signal cannot be prevented.
Also, with the conventional techniques, in the cases of an image containing an excessively high level of image information such that saturation of the output obtained from the image sensing operation is reached, a processed image is outputted in which a bright region has been rendered white such that its details become imperceptible. However, in the cases of the medical cameras, such as endoscope cameras, if correct chrominance information is not outputted, the problems occur in that the efficient and accurate diagnosis of an illness cannot be made. Therefore, the signal processing, in which the chrominance information is eliminated, is not appropriate for medical cameras.
The primary object of the present invention is to provide a color image processing method, wherein a false color phenomenon at a highlight area or a high-saturation area is capable of being minimized or prevented regardless of a luminance signal level.
Another object of the present invention is to provide an apparatus for carrying out the color image processing method.
The present invention provides a color image processing method, comprising the steps of:
i) performing an image sensing operation for detecting an image of an object,
ii) obtaining a plurality of color signals, which carry color image information of the object, in accordance with an output having been obtained from the image sensing operation, and
iii) performing a first signal suppressing process in which, in cases where a signal level of a certain color signal among the plurality of the color signals is higher than a predetermined signal level, the certain color signal is suppressed,
wherein the improvement comprises the steps of:
a) controlling the image sensing operation such that an image sensing operation with a comparatively large exposure quantity and an image sensing operation with a comparatively small exposure quantity are performed alternately,
b) detecting that suppression of a certain color signal among the plurality of the color signals is performed in the first signal suppressing process when the image sensing operation with a comparatively large exposure quantity is performed,
c) calculating composition ratios of the plurality of the color signals, which are obtained from the image sensing operation with a comparatively small exposure quantity, and
d) performing a second signal suppressing process for suppressing at least one color signal other than the certain color signal in accordance with an output, which has been obtained from the detection of suppression and which represents that the suppression of the certain color signal is performed, the second signal suppressing process being performed such that the composition ratios of a plurality of color signals, which are obtained from the second signal suppressing process, become identical with the calculated composition ratios.
In the color image processing method in accordance with the present invention, the image sensing operation is controlled such that the image sensing operation with a comparatively large exposure quantity and the image sensing operation with a comparatively small exposure quantity are performed alternately. Specifically, a single image sensing operation with a comparatively large exposure quantity and a single image sensing operation with a comparatively small exposure quantity may be performed alternately. Alternatively, several image sensing operations with a comparatively large exposure quantity and a single image sensing operation with a comparatively small exposure quantity may be performed alternately. No limitation is imposed upon the order in which the image sensing operation with a comparatively large exposure quantity and the image sensing operation with a comparatively small exposure quantity are performed.
The detection of suppression may be performed before the suppression is performed in the first signal suppressing process. Alternatively, the detection of suppression may be performed after the suppression has been performed in the first signal suppressing process.
The term xe2x80x9ccomposition ratios of a plurality of color signals become identical with calculated composition ratiosxe2x80x9d as used herein means that the composition ratios of the color signal of the same color as that suppressed in the first signal suppressing process and the other color signals, which are obtained from the second signal suppressing process, become identical with the calculated composition ratios. In such cases, all of the other color signals need not necessarily be suppressed such that the composition ratios of all of the plurality of the color signals become identical with the calculated composition ratios. Specifically, only the color signal, which causes the color reproducibility to be affected adversely due to the suppression of the certain color signal performed in the first signal suppressing process, may be suppressed such that its composition ratio may become identical with the calculated composition ratio, and the composition ratios of all of the color signals other than the certain color signal need not necessarily become identical with the calculated composition ratios. However, the composition ratios of all of the plurality of the color signals should preferably become identical with the calculated composition ratios.
Also, as described above, the composition ratios of the plurality of the color signals should become identical with the calculated composition ratios. Specifically, the composition ratios of the plurality of the color signals need not necessarily become exactly identical with the calculated composition ratios. It is sufficient for the composition ratios to be obtained such that the visual adverse effects upon the color reproducibility can be restricted by the suppression of the other color signals. The term xe2x80x9cidenticalxe2x80x9d as used herein has the meaning including the composition ratios in such cases. For example, the differences among the color signals may be calculated from the calculated composition ratios, and at least one of the other color signals may be suppressed such that the differences may be kept unchanged. In this manner, the visual color reproducibility may be prevented from becoming bad.
The color image processing method in accordance with the present invention should preferably be modified such that the image sensing operation with a comparatively large exposure quantity is performed by performing exposure for a comparatively long time, and the image sensing operation with a comparatively small exposure quantity is performed by performing exposure for a comparatively short time.
The present invention also provides an apparatus for carrying out the color image processing method in accordance with the present invention. Specifically, the present invention also provides a color image processing apparatus, comprising:
i) image sensing means for performing an image sensing operation for detecting an image of an object,
ii) color signals separating means for obtaining a plurality of color signals, which carry color image information of the object, in accordance with an output having been obtained from the image sensing means, and
iii) first signal suppressing means for performing a first signal suppressing process in which, in cases where a signal level of a certain color signal among the plurality of the color signals is higher than a predetermined signal level, the certain color signal is suppressed,
wherein the improvement comprises the provision of:
a) image sensing operation control means for causing the image sensing means to alternately perform an image sensing operation with a comparatively large exposure quantity and an image sensing operation with a comparatively small exposure quantity,
b) detection means for detecting that suppression of a certain color signal among the plurality of the color signals is performed by the first signal suppressing means when the image sensing operation with a comparatively large exposure quantity is performed,
c) composition ratio calculating means for calculating composition ratios of the plurality of the color signals, which are obtained from the image sensing operation with a comparatively small exposure quantity, and
d) second signal suppressing means for performing a second signal suppressing process for suppressing at least one color signal other than the certain color signal in accordance with an output, which has been obtained from the detection means and which represents that the suppression of the certain color signal is performed, the second signal suppressing process being performed such that the composition ratios of a plurality of color signals, which are obtained from the second signal suppressing means, become identical with the calculated composition ratios.
In the color image processing apparatus in accordance with the present invention, the image sensing operation control means should preferably control such that the image sensing operation with a comparatively large exposure quantity is performed by performing exposure for a comparatively long time, and the image sensing operation with a comparatively small exposure quantity is performed by performing exposure for a comparatively short time.
With the color image processing method and apparatus in accordance with the present invention, in cases where a certain color signal is suppressed in the first signal suppressing process, which has the same functions as those of a signal suppressing process performed in conventional techniques, when the image sensing operation with a comparatively large exposure quantity is performed, at least one color signal other than the certain color signal is suppressed in the second signal suppressing process such that the composition ratios of a plurality of color signals, which are obtained from the second signal suppressing process, become identical with the composition ratios of the plurality of the color signals, which are obtained from the image sensing operation with a comparatively small exposure quantity. Therefore, the false color phenomenon can be prevented perfectly, and the color reproducibility can be enhanced.
Also, with the color image processing method and apparatus in accordance with the present invention, reference is made to the composition ratios of the plurality of the color signals, which are obtained from the image sensing operation with a comparatively small exposure quantity. Therefore, in cases where the image contains an excessively high level of image information such that, if the image sensing operation with a comparatively large exposure quantity is performed, the output obtained from the image sensing operation saturates, the image sensing operation with a comparatively small exposure quantity is performed, and the composition ratios can be calculated accurately from the color signals, which do not saturate. Accordingly, the color reproducibility can be enhanced over a wide dynamic range.
Further, since the false color phenomenon can be prevented over a wide range, the HLCS processing becomes unnecessary, and appropriate chrominance information can be outputted in every case. Therefore, the color image processing method and apparatus in accordance with the present invention are suitable for medical cameras, such as endoscope cameras.
Furthermore, in cases where the exposure quantity is controlled by altering the exposure time, the exposure quantity can be changed by shutter speed control, in which driving pulses for driving the CCD image sensor are controlled. Therefore, the image sensing operation with a comparatively large exposure quantity and the image sensing operation with a comparatively small exposure quantity can be alternately performed easily.